Three-dimensional (3D) displays simultaneously generate multiple views per update of the display to create a 3D effect when viewed by the human visual system. Current consumer-grade augmented reality (AR)/virtual reality (VR) headsets, for example, typically generate two views, one for each eye, though more advanced AR/VR headsets may generate more views. Other examples of multi-view displays include, but are not limited to lenticular, multi-depth plane, volumetric and light-field displays.
A graphical display can be termed autostereoscopic when the work of stereo separation is done by the display so that the observer need not wear special eyewear to perceive the 3D effect. Light-field autostereoscopic 3D displays, for example, create the 3D hologram by presenting many perspective correct views simultaneously.
Examples of techniques for producing holograms can be found in U.S. Pat. No. 6,330,088, entitled “Method and apparatus for recording one-step, full-color, full-parallax, holographic stereograms” and naming Michael A. Klug et al. as inventors, (the “'088 patent”). U.S. (Publication 2008/0144174), entitled “Dynamic Autostereoscopic Displays,” published on Jun. 19, 2008 (the “'005 application”) and U.S. Pat. No. 8,736,675, entitled “Multi-Core Processor Architecture for Active Autostereoscopic Emissive Displays,” filed Dec. 3, 2007 (the “'675 patent”) further describe example light-field displays that present different views from different zones of the display to generate a dynamic 3D hologram that is visible to the unaided eye. The '088 patent, '005 application and '675 patent are hereby incorporated by reference herein in their entireties.
The processing of information to display a 3D image generally requires substantial amounts of computational effort to simultaneously render multiple images. As would be appreciated by those in the art, to render an image of a 3D model/scene, geometric data such as vertex positions and normal vectors can be transformed via vertex transformations and primitive assembly operations in the graphics pipeline before rasterization. More particularly, rendering an image may comprise rendering triangles. To render a triangle, the vertices are converted by a series of linear vertex transformations from their original 3D coordinate system to a 2D location and depth in image pixels.
3D multi-view displays typically rely on GPUs to carry out the rendering operations. While GPUs are powerful and effective processors for rendering large framebuffers from a single point of view, they suffer shortcomings when used with multi-view displays. In particular, conventional graphics pipelines implemented by GPUs are not well suited to multi-view rendering.
This is, at least in part, because the typical GPU graphics pipeline requires a separate render pass for each viewpoint/viewport required for the display system. Consequently, the host application or display render system providing 3D scene data to the GPU must repeatedly dispatch the scene to the GPU with a new viewpoint and viewport for each viewpoint/viewport change.
For example, to display a 3D image on a light-field display, a host application or light-field display render system may have to dispatch a scene to the GPU with a new viewpoint and viewport for each active hogel for which the GPU renders. The GPU, using a typical GPU graphics pipeline, will rasterize the scene for one hogel at a time, despite the fact that only the viewpoint and viewport change and not the scene description. Multi-view rendering on a GPU in this manner becomes a highly serial process by which vertex draw calls from the render engine and the resulting dispatch/transforms required per rendered view dominates the render pipeline.
Another issue with prior multi-view systems is that the host application and or render engine must know how many views a particular display supports in order to dispatch the correct viewpoints and viewports and to properly allocate viewports in the frame buffer. As such, the host and/or render engine will require new drivers to support each different type of display.
Moreover, displays that perform multi-view rendering often require much more time to complete a display frame than the host application. This requires that the host application stall and synchronize with the rendering system. Often this situation reduces the frame rate of the host application and sacrifices host application interactivity.
Multi-viewpoint rendering becomes a responsibility of the host application which must cache the render commands and then re-render the list for each viewpoint. Therefore, the host application must understand the exact nature of every view rendered for the display as well as any distortion corrections for any display specific optical properties.